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Optimal control of the state statistics 
for a linear stochastic system 

Yongxin Chen, Tryphon Georgiou and Michele Pavon 


Abstract —We consider a variant of the classical linear quadratic 
Gaussian regulator (LQG) in which penalties on the endpoint state 
are replaced hy the specification of the terminal state distrihution. 
The resulting theory considerably differs from LQG as well as from 
formulations that bound the probability of violating state constraints. 
We develop results for optimal state-feedback control in the two cases 
where i) steering of the state distribution is to take place over a finite 
window of time with minimum energy, and ii) the goal is to maintain the 
state at a stationary distribution over an inlinite horizon with minimum 
power. For both problems the distribution of noise and state are Gaussian. 
In the first case, we show that provided the system is controllable, 
the state can be steered to any terminal Gaussian distribution over 
any specified finite time-interval. In the second case, we characterize 
explicitly the covariance of admissible stationary state distributions that 
can be maintained with constant state-feedback control. The conditions 
for optimality are expressed in terms of a system of dynamically coupled 
Riccati equations in the finite horizon case and in terms of algebraic 
conditions for the stationary case. In the case where the noise and control 
share identical input channels, the Riccati equations for finite-horizon 
steering become homogeneous and can be solved in closed form. The 
present paper is largely based on our recent work in (Tj, (2) and presents 
an overview of certain key results. 

Keywords: Linear stochastic systems, stochastic optimal control, 
stationary distributions, covariance control. 


I. Prologue 

Consider the controlled evolution 

dx^{t) = A{t)x^ (t)dtt + B{t)u{t)dt + Bi{t)dw{t), 

x"(0) = xo a.s. (1) 

where xo is an n-dimensional Gaussian vector with density 

po(x) = (271)“"'''^ det(Eo)“^^^ exp (2) 

and independent of the standard p-dimensional Wiener process 
{tu(f) I 0 < f < r}; A, B and Bi are continuous matrix functions 
of t taking values in and respectively, Eq is a 

symmetric positive definite matrix, and T < oo represents the end 
point of a time interval of interest. 

The basic paradigm of Linear Quadratic Regulator Theory is 
to specify a target value for the state vector, e.g., the origin for 
simplicity, and impose the quadratic penalty 

Jt{x{T)) ■=R{x{T)'Mx{T)} 

on possible deviation. Thence, the sum of the quadratic norm of the 
input 

J(u):=E|y u{t)'u{t) dt}^ 
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and the terminal cost is sought to be minimized over possible control 
laws. The solution is well known, see e.g. n, di Section II]. It can 
obtained by solving the matrix Riccati equation 

11(f) = -A(f)'n(t) - n{t)A{t) + n(f)B(f)B(f)'n(f) o) 

for {n(f) I 0 < f < r} with boundary condition n(T’) = M. Here, 
M is a symmetric matrix which is typically taken to be positive semi- 
definite, but it does not need to. The variational analysis shows that 
the optimal feedback control law is 

u{x,t) ■= —B(t)'ll{t)x (4) 

and the optimal cost is 

j = j{vr) + jT{x*{T)) 

— trace(Eon(0)) + / tTa.ce{Bi(t)B\(t)'l\.{t))dt. 

Jo 

The terminal state x*{T) of the controlled process 

dx*{t) = {A(t) — B{t)B{tyil{t)) X* {t)dt + Bi(t)dw{t), 

a;*(0) = Xo a.s. (5) 

has probability density 

Pt{x) = (27r)“"^^ det(ET)~^^^ exp ^—^x'T,^^x'^ , (6) 

where the terminal covariance Er = E(r) is obtained from 

E(f) = [A{t) - B{t)B{tyU{t)) E(f) 

+ E(f) {A{t) - B{t)B{tyU{t)y + B4t)Bi{ty (7) 

with boundary condition E(0) = Eq. The standard formalism of 
LQG does not directly address terminal conditions which are only 
indirectly impacted by the choice of the “weight” M. In the present 
work, we study the question of what values of the state covariance 
E(T) can be obtained through feedback control. Moreover, for a 
given covariance, we consider the problem to steer the system to the 
corresponding state-distribution with minimal (quadratic) effort for 
the control input. 

Turning to a time-invariant system, that is Q with A, B, B\ 
being independent of time, any state-feedback gain K for which A — 
BK is a Hurwitz matrix corresponds to a state covariance E for the 
stationary controlled state process. It is well known that E satisfies 
the Lyapunov equation 

(A - BK)T, -f E(A - BK)' + BiB) = 0. 

In this context we are interested in characterizing positive definite 
matrices that are admissible as stationary state-covariances, and to 
determine a choice for the feedback gain K (assuming that there is 
more than one) that minimizes power at the control input. 

Both of these problems, to steer and possibly maintain the state 
statistics of a stochastically driven system, represent variants of the 
classical regulator problem which is at the heart of many control 
applications and entails efficient and accurate steering to a target 
location, see e.g. m, m, n, □. The paradigm that is put forth 
in this paper represents a “relaxed” version of the classical linear 
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quadratic regulator (LQG) in that hard constraints and penalties on 
the endpoint state, are replaced by “soft conditioning” on the state to 
be distributed according to a prescribed probability density. This for¬ 
mulation appears natural for applications in quality control, industrial 
and manufacturing processes. Further applications may be envisaged 
in the control of aircrafts, UAVs, and autonomous cars, where speci¬ 
fications on the state distribution are also natural. Further motivation 
for this work is provided by modern technological advances that 
allow us to manipulate micro and thermodynamic systems, and to 
measure physical properties with unprecedented accuracy. Many such 
advances rely heavily on our ability to limit state-uncertainty using 
feedback, e.g., in oscillators coupled to a heat bath or in steering the 
collective behavior of a swarm of particles experiencing stochastic 
forcing. Cutting edge examples include thermally driven atomic force 
microscopy, the control of molecular motors, laser driven reactions, 
the manipulation of macromolecules, the “active cooling” of devices 
aimed at measuring gravitational waves, and the focusing of particle 
beams (see ©, g), (TO), GD, QD, fTSl l. 

In order to provide insight on our ability to steer and maintain 
the probability density of the state, it is useful to recall analogous 
results regarding controllability of a deterministic system. More 
specifically, consider 

x{t) — A{t)x{t) + B{t)u{t), t £ [0, oo) (8) 

with as before, x{t) £ ffi”, u{t) £ R™, and the problem 

to steer from the origin to a given point x{T) = ^ £ R". This 
is of course possible for any arbitrary ^ £ R" iff the system is 
controllable, i.e., the controllability Gramian 

f r)B(r)B(r)^<l?(f, r)^dr 

Jo 

is positive definite for all f > 0, or in case A, B are constant, the rank 
of [B, AB,..., A"'~^B\ is n; these will be standing assumptions 
throughout. It is well known that the steering can be effected in a 
variety of ways, including “minimum-energy” control, over any pre¬ 
specified interval [0, T\. On the other hand, for the case where A, B 
are time invariant, maintaining a fixed value ^ for the state vector in 
a stable manner is not always possible. For this to be the case, using 
feedback and feedforward control, the equation 

0 = (M - BK)£, + Bu (9) 

must have a solution (u, K) for a constant value of the input u 
and a corresponding value for K so that A — BK is Hurwitz (i.e., 
the feedback system be asymptotically stable). This reduces to the 
requirement that 

0 = -I- Bv 

for some v\ if there is such a v, we can always choose a suitable 
K so that A — BK is Hurwitz and then, from v and K, we can 
compute the constant value u. Conversely, from u and K we can 
obtain v = u — K^. 

A similar dichotomy between our ability to steer the distribution 
of the state vector over a finite window in time, and to maintain 
an admissible stationary distribution over the infinite time-horizon 
exists. This is explained next. In what follows, we focus on achieving 
the control objective with minimal energy and power, respectively, 
and consider these questions in the case of Gaussian distributions 
for the noise and state vector. It will be shown that the state- 
covariance can be assigned at the end of an interval through suitable 
feedback control if and only if the system is controllable. On the 
other hand, a positive semidefinite matrix is an admissible stationary 
state-covariance attained through constant feedback if and only if it 
satisfies a certain Lyapunov-like algebraic equation. 


The ability to specify the mean value of the state-vector reduces 
to the deterministic problem just discussed. More specifically, since 
E{a;(f)} =: x{t) satisfies ID, controllability or the system is 
necessary and sufficient to specify x{T) at the end of the interval and 
this is effected by a deterministic mean value for the input process. 
Likewise, the mean value for a stationary input must satisfy to 
attain x{t) = ^ for a stationary state process. Thus, throughout and 
without loss of generality we assume that all processes have zero- 
mean and we only focus on our ability to assign the state-covariance 
in those two instances. 


IT Finite horizon steering 

Let Et be symmetric and positive definite n x n matrix 
.Consider the “target” Gaussian end-point distribution 

Pt{x) = (27r)“"^^ det(ET)~^^^ exp ^x'Ey^a;^ , (10) 

for G)- We seek to determine whether is possible to steer Q from the 
initial probability density po to this “target” final probability density 
Pt and, if so, to do this optimally by minimizing 

J(M);=E|y u{t)'u{t)dt^ <oo, (11) 

over adapted, ^finite-energy control functions such that has a 
strong solution on [0,r] and x“(T) is distributed according to l |10[ >. 
To this end, we let U represent the family of all such admissible 
control laws and consider the following. 

Problem 1: Determine u* that minimizes J(u} over all u £U, 
i.e., over adapted inputs that steer the system from state-covariance 
Eq to Et. 

As it will be apparent in what follows, our results also cover 
the case where one or both endpoint marginal densities are delta 
functions. If Pt(x) = 5{x — xq) , the optimal control becomes 
unbounded in suitable directions as f T to push the whole 
diffusion into xo as in the Brownian bridge. Though in the limit, 
the optimal control may fail to have finite-energy (see also [Ml). 


A. Conditions for optimality 

Clearly, if {n(f) | 0<f<T}isa solution of the matrix 
Riccati equation 0, u{x, t) given by |^, and it is such that the 
end value x* (T) of the Gauss-Markov process in 0 has probability 
density pr, then u{x*{t),t) is indeed the solution u* to Problem[^ 
We recast this observation as follows (Proposition [^. 

First, let E(f) be the state covariance of 0 , i.e., a solution of 
0, which therefore satisfies the two boundary conditions 

E(0) = Eo, E(r) = Et. (12) 

Then, define 

H(f) := E(f)~^ -n(f) 

(note that E(f) is positive definite on [0, T] since, by assumption, 
Eo is already positive definite. A direct calculation using 0 and 0 

^I.e., u E is such that u{t) only depends on t and on {a:^(s) | 0 < s < 
for each t G [0, T] 
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leads to ( fT3bl > below. We therefore arrive at a nonlinear system of 


equations 

n = -A'n-IlA + IlBB'n (13a) 

H = -A'¥L-HA-RBB'¥L (13b) 

+ (n + H) {BB' - BiB[) (n + H) 

Eo^ = n(0) + H(0) (13c) 

Ey^ = n(r) + H(r). (i3d) 


The case n(f) = 0 corresponds to the situation where the uncon¬ 
trolled evolution already satisfies the boundary marginals and, in that 
case, is simply the prior state covariance. We summarize as 

follows. 

Proposition 1: Assume that {(n(f),H(f)) | 0 < f < T} satisfy 
OH- OH- Then the feedback control law 0 is the solution to 
Problem [T] and the corresponding optimal evolution is given hy 0 . 

For the special case where B{t) = B\{t), that is, the case where 
the control inputs and the noise enter the system through identical 
channels, the nonlinear system of equations l |13[ ) reduces to|^ 


n 

= -A'H - HA + HBB'H 

(14a) 

H 

= -A'H - HA - HBB'H 

(14b) 


= n(o) + H(o) 

(14c) 


= n(r)-FH(T). 

(14d) 


For this simplified set of equations ou the existence and 
uniqueness of solutions follows already from works by Fortet CD, 
Beurling |[l7], Jamison (Tsj. Fbllmer (B) Indeed, in this case the 
problem reduces to a special linear-quadratic version of the so- 
called Schrodinger bridge problem. It should be noted that, while 
Schrddinger bridges CD are constructed for general diffusions, they 
are restricted in two essential ways. First the diffusion coefficient is 
nonsingular (corresponding to B\ being square and invertible) and 
then, they correspond to the case where control enters in precisely 
the same way as the noise tH (B = Bi). 

Herein, we are interested in the general case where control and 
noise excitation channels may differ, albeit, we restrict our attention 
to linear dynamics and Gaussian statistics. In (T) we considered l |I4^ 
directly and provided a solution in closed-form. This solution (T) 
Proposition 4], which is not repeated here due to space limitations, 
provides a direct proof of existence of solutions for GD without 
relying on the theory of the Schrodinger bridges. In particular, it 
establishes/eflJifoi/ify of Problem i.e. that U is nonempty and that 
there exists a minimizer for this special case where B{t) = B\(t). 
In general, when B(t) 7 ^ B\{t) (and time-varying) it is not known 
whether Problem[T]has a minimizer. However, when B and Bi do not 
depend on time, it turns out that the set of admissible controls U is 
not empty. This is shown helow and in Section [Il-C| we also provide 
an approach that allows constructing suboptimal controls incurring 
cost that is arbitrarily close to J{u)- 


B. Controllability of state statistics 

We now turn to the “controllability” question of whether there 
exist admissible controls to steer the controlled evolution Q to a 
target Gaussian distribution Et at the end of a finite interval [0, T]. 

^This corresponds to a so-called Schrddinger system QU, consisting of 
a forward and a backward Kolmogoroff (partial) differential equation that 
are coupled through their boundary conditions. Since the distributions are 
Gaussian, the system entails matrix differential equations. 


We do so for the case where A,B and Bi, are time-invariant, and 
to this end, we search over controls that can be expressed in state- 
feedback form 

u{x,t) ~—K{t)x. (15) 

Then, the state covariance 

E(f) := E{a;(f)a:(f)'} 

of (|T]l satisfies the Lyapunov differential equation 

t{t) = {A- BK{t))T.{t) + E(f)(A - BK{t)y + BiB[ (16) 

with E(0) = Eq. Regardless of the choice of K{f), GD specifies 
dynamics that leave invariant the cone of positive semi-definite 
symmetric matrices 

5+ — {E I E G E = E' > 0}. 

To see this, note that the solution to ( |16t is of the form 

E(f) = 4>(L0 )Eo-I>(L0)'-F [ l>(Lr)BiB;l>(Lr)'dr 

where ^(t, 0) satisfies 

and 6(0,0) = I, the identity matrix; i.e., 6(f, 0) is the state- 
transition matrix of the system x(t) = (A — BK{t))x{t). 

Our interest is in our ability to specify E(T') via a suitable 
choice of K{t). To this end, define 

Uit) := -E(f)JT(f)', 

and observe that U{t) and K{t) are in bijective correspondence 
provided that E(f) > 0 (which follows from Eq > 0). Thus, we 
now consider the differential Lyapunov system 

E(f) = AE(f) -F E(f)A' -F BU{t)' -F U{t)B'. (17) 

Reachability/controllability of a differential system such as 0, or 
G3’ is the property that with suitable bounded control input u{f), 
or U{f), respectively, the solution can be driven to any finite value. 
Interestingly, if either 0 or l|17[ > is controllable, so is the other. But, 
more importantly, when (|17t is controllable, the control authority 
allowed is such that steering from one value for the covariance to 
another can be done by remaining within the non-negative cone. This 
is stated as next (see Theorem 3]). 

Theorem 2: The Lyapunov system G3 is controllable iff 
{A, B) is a controllable pair. Furthermore, if l |17[ ( is controllable, 
given any two positive definite matrices Eq and Et and an arbitrary 
Q > 0, there is a smooth input Uf) so that the solution of the 
(forced) differential equation 

t{t) = AE(f) -F E(f)A' -F BU{t)' + U{t)B' + Q (18) 

satisfies the boundary conditions E(0) = Eq and E(r) = Et and 
E(f) > 0 for all t £ [0,T]. 

Remark 3: Although the result on the controllability of state 
covariance we establish is for time-invariant system, we believe that 
a similar result holds for time-varying system. 
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C. Finite interval minimum energy steering of state statistics 

We are interested in computing an optimal choice for feedback 
gain K{f) so that the control u{f) = —K{t)x{f) steers 0 from an 
initial state-covariance Eq at f = 0 to the final St at t = T. The 
expected control-energy functional 

J{u) := ® u{tyu{t)dt^ (19) 

= f trace{K {t)'E{t)K{ty)dt 
Jo 

needs to be optimized over K {t) so that as well as the boundary 
conditions 

E(0) = So, and E(r) = St (20 a) 

hold. 


If instead of the choice K(t) we sought to optimize over 
U{t) := —T,{f)K{ty and E(f), the functional l |19[ > becomes 

J = [ tTace{U{ty'£{t)~^U{t))dt 
Jo 

which is jointly convex in U{t) and E(f), while cn is replaced by 
t{t) = ^E(f) -f E(f)A' -h BUit)' + U{t)B' + BiB[ (20b) 


which is now linear in both! As a consequence, the optimization can 
be written in the form of a semi-definite program to minimize 


trace(y (t))dt 


subject to l|20a|20b| and 


Y{t) 

U{t) 


u{ty 

m 


> 0 . 


(20c) 


(20d) 


This can be solved numerically after discretization in time, and 
a (suboptimal) gain recovered using the correspondence K{f) = 
-[/(f)'E(f)-i. 


from what follows, even when the problem is feasible, i.e., there 
exist controls which maintain E, an optimal control may fail to 
exist. The problem has connections to Jan Willems’ classical work on 
the Algebraic Riccati Equation (201 and this is discussed following 
Proposition 


A. Condition for optimality 

Let us start by observing that the problem admits the following 
finite-dimensional reformulation. Let K, be the set of all m x n 


matrices K such that the corresponding feedback matrix A — BK is 
Hurwitz. Observe that 

E{uu} = E{x'K'Kx} = trace(A'EjT') 

Then Problem reduces to finding am x n matrix K £ K. which 
minimizes 

JiK) = trace {KEK') (24) 

subject to the constraint 

{A - BK)E + E(A' - K'B') + BiB[ = 0. (25) 

Consider the Lagrangian 

JZ{K,Il) = trace {KEK') (26) 

-h trace (n((A - BK)E + S(A' - K'B') + BiB[)) . 

Note that since K. is open, a minimum point may fail to exist. 
Standard variational analysis (2) leads to the form K = B'Tl for 
the optimal gain. We summarize this as follows. 


Proposition 2: Assume that there exists a symmetric matrix 11 
such that A — BB'Il is a Hurwitz matrix and 

{A - BB'n)E + E(A - BB'ny -f BiB[ = 0 (27) 

holds. Then, the solution to Problem]^ is 

u{f) = -B'nx{t) (28) 


III. Infinite-horizon steering 

When A, B and Bi do not depend on time, we seek a constant 
state feedback law u{t) = —Kx{t) to maintain a stationary state- 
covariance E > 0 for 0. In particular, we are interested in one that 
minimizes the expected input power (energy rate) 


We now draw a connection to some classical results due to 
Jan Willems l20l . In our setting, minimizing 0 is equivalent to 
minimizing 

Jpo^eriu) -\-E{xQx} (29) 

for an arbitrary symmetric matrix Q since the portion 
E{x'Qx} = tracejQE} 


Jpower(w) ;= (21) 

and thus we are led to the following problenj^ 

Problem 4: Determine u* that minimizes Jpower(rt) over all 
u{t) = —Kx{f) such that 

dx{t) = (A — BK)x{t)dt -I- Bidw{t) (22) 

admits 

p{x) = (27r)~"''^^ det(E)”^'^^ exp ^—^x''E~^x^ (23) 

as invariant probability density. 


In general, this problem may not have a solution. In Theorem 
we provide conditions that ensure E is admissible as a stationary 
state covariance for a suitable input. Moreover, as it will be apparent 


^ An equivalent problem is to minimize limT—>oo 
for a given terminal state covariance as T —>■ oo. 


Ee I u{tyu{t)dt^ 


is independent of the choice of K (because E is given). On the other 
hand, minimization of (29[ > for specific Q, but without the constraint 
that Ejata:'} = E, was studied by Willems 1201 and is intimately 
related to the maximal solution of the Algebraic Riccati Equation 
(ARE) 

A'n + TIA- iiBB'n + g = 0. (30) 


Specifically, under the assumption that the Hamiltonian matrix 


H = 


A 

-Q 


-BB' 

-A' 


has no pure imaginary eigenvalues, Willems’ result states that A — 
BB'Ilis Hurwitz and that l |28^ is the optimal solution. Thus, starting 
from a symmetric matrix H as in Proposition|^ we can define Q using 

Q = -A'H - HA + TlBB'n. 


By Willems’ results, a has at most one “stabilizing” solution 
H, and therefore, the matrix in the proposition coincides with the 
maximal solution to ( [30^ . Therefore, if Problem [4] has a solution, this 
corresponds to the maximal solution of the AREl[30|l for a particular 
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choice of Q. Interestingly, neither 11 nor Q, corresponding to an 
optimal control law for which \21\ holds, are unique, whereas K is. 
The computation and the uniqueness of the optimal gain K will be 
discussed later on in Section UlI-CI 


B. Characterization of stationary state statistics 

We now focus on what values of the state covariance are 
admissible in that they can be obtained by state feedback. 

The constraint ( |25t already implies that 

AT. + T.A'+ BiB[+BX'+ XB'= 0 (31a) 

can be solved for X. 

In particular we can take X = —TiK' in l |25[ ). Thus, the solvability 
of l |31a| l is a necessary condition for E to qualify as a stationary 
state-covariance attained via feedback. This condition, ( |31a[ (, can be 
written as a rank condition: 


AE -f EA' -h HiBj 

B 

= rank 

'0 

B 

B 

0 


B 

0 


which ensures that AS + SA' -|- BiB[ is in the range of the linear 
map X I— 1 - BX' + XB', cf. 1211 Proposition 1], 

Conversely, given E > 0 that satisfies and a solution X of 
l |31a^ , then j25[ l holds with K = —X'S~^. Therefore, provided A — 
BK is a Hurwitz matrix, E is an admissible stationary covariance. 
Now, the property of A — BK being Hurwitz can be guaranteed 
when {A — BK, f3i) is a controllable pair. In turn, controllability of 
{A — BK, Bi) is guaranteed when TZ{B) C TZ{Bi). Thus, we have 
established the following. 

Theorem 5: Consider the Gauss-Markov model Q and assume 
that TZ{B) C TZ{Bi). A positive-definite matrix E can be assigned as 
the stationary state covariance via a suitable choice of state-feedback 
if and only if E satisfies any of the equivalent statements l |31a|31b| . 

Holtz and Skelton 1221 considered the problem to maintain a 
stationary state covariance with feedback and provide as condition 
that AS + SA' -I- BiB[ is in the null space of the operator Y i—>■ 

■ It turns out that this condition is equivalent to 
l |31a|31b| l, see (S). On the other hand, conditions 131a|31b| l were 
obtained in ED, ED, for the special case when B — Bi, as being 
necessary and sufficient for a positive-definite matrix to materialize 
as the state covariance of the system driven by a stationary stochastic 
process (not-necessarily white). 


C. Minimum energy control 

As we just noted, a positive definite matrix E is admissible as 
a stationary state-covariance provided ( |31a[ > holds for some X and 
A + BX'S~^ is a Hurwitz matrix. The condition TZ{B) C TZ{Bi) 
is a sufficient condition for the latter to be true always, but it may 
be true even if TZ{B) C TZ{Bi) fails (see Example 1 in Section [Tv^ . 
Either way, the expected input power (energy rate) 

E{m'w} = trace(A'EA'') (32) 

= trace(X'E“^X) 

in either K, or X. Thus, assuming that TZ{B) C TZ(Bi) holds, and 
in case ( |31a[ l has multiple solutions, the optimal constant feedback 
gain K can be obtained by solving the convex optimization problem 

min {trace(X^E”^X) | ( |31a[ ( holds } . (33) 


Remark 6: In case TZ{B) % TZ{Bi), the condition that A—BK 
be Hurwitz needs to be verified separately. If this fails, we cannot 
guarantee that E is an admissible stationary state-covariance that can 
be maintained with constant state-feedback. However, it is always 
possible to maintain a state-covariance that is arbitrarily close. To 
see this, consider the control 

K, = K + ^eB'S~^ 

for e > 0. Then, 

{A - BK,)S + S{A- BK,y = -cBB'- BiB[ 

< -cBB'. 

The fact that A — BKe is Hurwitz is obvious. If now Ee is the 
solution to 

{A- BK,)S, + S4A- BK,y = -BiB[ 
the difference A = E — E^ > 0 and satisfies 

{A- BK,)A + A{A- BK,y = -cBB', 

and hence is of o(e). 


TV. Examples 


Example I 


Consider particles that are modeled by 

dx(t) = v(t)dt + dw{t) 
dv{t) = u{t)dt. 

Here, u{t) is the control input (force) at our disposal, x{t) represents 
position and v{t) velocity (integral of acceleration due to input 
forcing), while w{t) represents random displacement due to impulsive 
accelerations. Alternatively, J* v[T)dT may represent actual position 
while x{t) noise measurement of position. 

The purpose of the example is to highlight a case where the 
control is handicapped compared to the effect of noise. Indeed, the 
displacement w{f) directly affects x(f) while the control effort u{t) 
needs to be integrated before it mitigates the effect of wit) on the 
position x(f) of the particles. 


Another interesting fact that this example highlights is that 
TZ{Bi) is not necessary for being able to ensure that, 
particular positive definite E’s in the range of A i—>■ BX' + XB', 
can be maintained as stationary state covariances. Indeed, here, 
B-(B) 2 TZ{Bi) since B = [0, 1]' while Bi = [1, 0]'. Yet, if 
we choose 


1 - 1/2 

- 1/2 1/2 


(34) 


as a candidate stationary state-covariance, it can be seen that j31a| l 
has a unique solution X giving rise to TV = [1, 1] and a stable 
feedback since A — BK is Hurwitz. 


We now wish to steer the spread of the particles from an initial 
Gaussian distribution with Eq = 27 at t = 0 to the terminal marginal 
El at f = 1, and from there on, since Ei is an admissible stationary 
state-covariance, to maintain with constant state-feedback control. As 
we just noted, this is indeed possible. Eigure[T] depicts typical sample 
paths in phase space, as a function of time, that are attained using 
i) the suboptimal feedback strategy derived following ( |20| l over the 
time interval [0, 1] and, ii) static state-feedback with K = [1, 1] over 
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Fig. 1: State trajectories (Example 1) Fig. 3: Steady state trajectories in phase space (Example 2) 



Eig. 2: Control inputs (Example 1) 


the time window [1, 2]. Eigure displays the corresponding control 
action for each trajectory over the complete time interval [0, 2], which 
consists of the “transient” interval [0, 1] to the target distribution and 
the “stationary” interval [1, 2]. 


Example 2 


once again, w{t) represents random displacement due to impulsive 
accelerations. We (arbitrarily) select 


S 


x,v 


'7/4 0 ■ 

0 3/4 


as a desirable steady-state covariance for the projection of the process 
onto the phase-plane of the particle dynamics (x,v). (The additional 
component Xc corresponding to the actuator is not shown.) Eirst, we 
need to determine whether is indeed an admissible steady-state 
covariance and, if so, to determine an optimal choice for the constant 
state-feedback gain that ensures the state is distributed accordingly. 
To this end, we seek a choice of a variance for Xc and of a 
cross-covariance Y between Xc and (ai, v) so that 


E = 




> 0 


is a admissible stationary state-covariance for |35). Eor this to be 
true, we need to verify that (|31b[( holds with 



0 1 o' 


'o' 


'o' 

A = 

0 0 1 

0 0-1 

, B = 

0 

1 

, Si = 

1 

0 

indeed the case for Y 

= [-3/4 


1/2]', E 

Xq 


3/4. 


Eor the above choice of E, the optimal gain and power are 
found to be TT = [1 3 2] and J = 5/2 by solving using e.g., 
m. Eor this solution, stationary state trajectories, projected onto the 
{x, u)-coordinates are now displayed in Eigurej^ 


We now consider a more detailed model of inertial particles 
where random acceleration as well as control are effected through 
forcing along the same channel. Yet, the control is again “handi¬ 
capped” due to lag in actuation dynamics. An additional aspect of 
the example is that it highlights a case where “target state statistics” 
may be prescribed for only a portion of the combined controller- 
system dynamics, namely, for the dynamics corresponding to the 
inertial particles and not the controller. 


As before the steering between specified Gaussian probability 
densities over an interval [0,1] follows Section II-C Eor complete¬ 
ness, we display in Figure sample paths corresponding to the 
transition between marginals with covariance matrices Eq = 3/ to 
El = E, respectively. The figure shows the projection onto the {x, v)- 
component of the process that corresponds to position and velocity. 


To this end, we consider the second-order process 

dx{t) = v{t)dt 
dv{t) = Xc{t)dt -f dw{t) 

dxc{t) = —Xc{t)dt + u{t)dt. (35) 

Here, Xc is the 1-dimensional state/output of the actuator and repre¬ 
sents force, u{t) represents the control signal to the actuator while. 


V. Epilogue 

In this paper, we have outlined some of the main results in Q), 
1^ . These results, in the finite horizon problem, may be viewed as a 
generalisation of the theory of Schrodinger bridges O for Gauss- 
Markov models to the situation where the noise does not affect all 
directions and where the control also acts through a different channel 
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Fig. 4: Finite-interval steering in phase space (Example 2) 

(recent attempts to address linear stochastic systems were also limited 
to non-degenerate diffusions where the control and noise channel 
are identical I25l . (26)). Although the theory is far from complete, 
our work has provided the first implementable form of the optimal 
control. Taking into consideration also the algorithmic solutions of 
the Schrddinger system provided in (27), we have now at disposal 
a new powerful tool to attack a variety of problems in classical and 
modem control including some applications in unexpected directions 
such as the optimal mass transport problem 1281 . (29). 
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